Travel control device

ABSTRACT

In the present invention, it is possible to accurately predict, at an earlier timing, that a pedestrian will perform constant speed movement or a route change that is more complex than the constant speed movement. Provided is a travel control device that can accurately determine a change in the route of the pedestrian according to a change in the pedestrian&#39;s posture and, in particular, a change in the orientation of the body or a change in an inverted angle, and that can appropriately control the travel of the vehicle.

TECHNICAL FIELD

The present invention relates to a device for recognizing a motion of apedestrian around a vehicle from an image obtained using image sensingmeans such as a camera built in the vehicle and controlling a vehicletravel on the basis of a result of the recognition.

BACKGROUND ART

Conventionally, there is known a control device for controlling avehicle travel by recognizing a condition around a vehicle from an imageobtained from image sensing means such as a camera built in the vehicle.For example, PTL 1 discloses a vehicle surrounding monitoring devicethat decides a posture of a pedestrian from an image around the vehicleto rapidly determine a pedestrian to be avoided from colliding with thevehicle and presents information to a driver or controls a behavior ofthe vehicle. For example, PTL 2 discloses a travel support method forcontrolling a vehicle travel by obtaining a change of a pace length of apedestrian, a both-foot stance time, or the like, and obtaining anpredicted travel route by estimating an abrupt stop of the pedestrian.

CITATION LIST Patent Literature

PTL 1: JP 2007-279808 A

PTL 2: JP 2009-012521 A

SUMMARY OF INVENTION Technical Problem

In the vehicle surrounding monitoring device described in PTL 1, theextracted posture of a pedestrian is decided to determine whether or notthe extracted pedestrian is a target to be avoided from colliding withthe vehicle, and the vehicle control is performed on the basis of aresult of this determination. However, in the collision avoidance methodof the vehicle surrounding monitoring device described in PTL 1, atravel route of the pedestrian is estimated from the currentpedestrian's posture. Therefore, when a speed of the pedestrian or adirection of the speed is changed, it is difficult to estimate thetravel route of the pedestrian.

In the travel support method described in PTL 2, information on pixelpatterns extracted from a photographic image in front of a host vehicleis obtained, and a decreasing chance of a pace length of a pedestrianapproaching in a travel route of the host vehicle, a both-foot stancetime, or the like is obtained, so that whether or not the pedestrianstops is determined. In addition, in order to avoid this travel route,an avoidance route is determined by setting this range as a travelinhibition range, and the travel of the host vehicle is controlled suchthat the host vehicle travels along the avoidance route. However,although whether or not a pedestrian stops is predicted on the basis ofa decreasing change of the pace length, it is difficult to predict achance of the speed direction of the pedestrian. Furthermore, as thepace length changes, in general, a walking speed also changes at thesame time. Therefore, using a change of the walking speed, that is,information on the acceleration of the pedestrian, the travel of thehost vehicle is controlled. For this reason, it is difficult to predicta travel route of a pedestrian, which is more complicate than a constantspeed movement.

The present invention was made to solve the above problems, and it is anobject of the present invention to provide a travel control devicecapable of appropriately controlling a vehicle travel by obtaining achange of a pedestrian's travel route with high accuracy.

Solution to Problem

In order to achieve the aforementioned object, the vehicle controldevice according to the present invention is characterized in that aspeed of a vehicle is changed depending on a change between current andpast pedestrian's body angles.

Advantageous Effects of Invention

Using the solving means according to the present invention describedabove, it is possible to provide a travel control device capable ofappropriately controlling a vehicle travel by obtaining a change of thepedestrian's travel route with high accuracy.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating main components and a flow ofinformation according to the present invention.

FIGS. 2(a) and 2(b) are diagrams illustrating a control method accordingto the present invention.

FIGS. 3(a) and 3(b) are diagrams illustrating a pedestrian's posturerecognition method according to the present invention.

FIGS. 4(a) and 4(b) are diagrams illustrating a control method accordingto the present invention.

DESCRIPTION OF EMBODIMENTS

Embodiments of the present invention will now be described withreference to the accompanying drawings. FIG. 1 is a diagram illustratingan embodiment of a travel control device according to the presentinvention. FIG. 1 is a block diagram illustrating a surroundingenvironment detection device 2, a vehicle control device 3, andperipheral devices included in the travel control device 1.

The surrounding environment detection device 2 is means for detecting asurrounding environment of a host vehicle and includes an externalenvironment recognition device 21 configured to obtain informationregarding an surrounding environment of the host vehicle, and asurrounding environment recognition unit 22 configured to determineexistence of an obstacle or identify a type or shape of the obstacle onthe basis of image data or an electric signal of the externalenvironment recognition device 21. For example, the external environmentrecognition device 21 is an on-vehicle camera for photographing asurrounding environment in front of the host vehicle. For example, theexternal environment recognition device 21 includes four on-vehiclecameras for photographing front, rear, left, and right surroundingenvironments of the host vehicle. The image data obtained from theon-vehicle camera is input to the surrounding environment recognitionunit 22. The external environment recognition device 21 may include aradar that measures a distance from an object using millimeter waves orlaser, a sonar that measures a distance from an object using ultrasonicwaves, or the like as well as the on-vehicle camera.

The surrounding environment recognition unit 22 detects a shape orposition of an object such as a solid object at rest around the hostvehicle, a mobile object, a road surface paint, or a sign using theimage data input from the external environment recognition device 21 oran electric signal from the radar or the like. In particular, thesurrounding environment recognition unit 22 has a functionality ofdetecting a pedestrian's body angle as posture detection means.

The solid object at rest includes, for example, a vehicle, a wall, apole, a curbstone, a building, or the like. In addition, the mobileobject includes, for example, a pedestrian, a vehicle, a bicycle, amotorcycle, or the like. Hereinafter, the solid object at rest and themobile object will be collectively referred to as an “obstacle.” Theshape or position of an object is detected using a pattern matchingmethod or other methods. The position of the object is expressed on thebasis of a coordinate system in which an origin is set to the positionof the on-vehicle camera that photographs a forward sight of the hostvehicle. For a pedestrian, information is detected, including aprojected area toward the on-vehicle camera, positions and directions ofa body, a bust, and a head, a line-of-sight direction, a grounded pointbetween a leg and the ground surface, and the like.

The external environment recognition device 21 outputs information on adistance from a recognized object, a shape or orientation of the object,and the like as well as the image data directly as analog data or ananalog-to-digital converted version to the vehicle control device 3through a dedicated line or the like.

The vehicle control device 3 illustrated in FIG. 1 is a computer forcontrolling the host vehicle and serves as a target route generator 31,a collision predictor 32, and a vehicle controller 33 by executing aprogram stored in a storage medium (not shown). Note that the vehiclecontrol device 3 has a memory unit 34. The memory unit 34 is capable ofstoring previous pedestrian's body angles.

The vehicle control device 3 is connected to the steering device 102,the driving device 103, and the braking device 104 of the host vehicle,and the surrounding environment detection device 2, the sound generatordevice 105, and the display device 106 provided in the host vehicle. Inaddition, the vehicle control device 3 is connected to an informationcommunication line CAN (not shown) of the host vehicle, so that vehicleinformation such as a speed of the vehicle or a steering angle of thehost vehicle is input through the CAN.

The steering device 102 includes an electric power steering or the likecapable of controlling a steering angle using an electric actuator orthe like in response to a driving command of the vehicle control device3. The driving device 103 includes an engine system, an electric powertrain system, or the like capable of controlling a driving force inresponse to a driving command of the vehicle control device 3. Thebraking device 104 includes an electric brake, a hydraulic brake, or thelike capable of controlling a braking force using an electric orhydraulic actuator or the like in response to a driving command of thevehicle control device 3. The sound generator device 105 includes aloudspeaker and the like and is used to output an alarm, voice guidance,or the like for a driver. The display device 106 includes a display of anavigation device or the like, a meter panel, a warning light, and thelike. The display device 106 displays an operation screen for thevehicle control device 3, a warning screen for visually notifying adriver of a fact that the host vehicle is in danger of collision with anobstacle, or the like.

The target route generator 31 generates a travel route for moving thehost vehicle from the current host vehicle position to a targetposition. For example, assuming that a vehicle travels on a public road,a destination is set using a navigation device or the like provided withmap data, and the route is generated from information such as apositional relationship between the host vehicle and the obstacle or atraffic lane location for traveling to the destination.

The collision predictor 32 determines whether or not the host vehicleand the obstacle will collide when the host vehicle travels along thetravel route generated by the target route generator 31. The collisionpredictor 32 predicts a travel route of the mobile object on the basisof a recognition result of the surrounding environment recognition unit22. A travel route of the mobile object, in particular, a travel routeof the pedestrian is predicted on the basis of a pedestrian's postureand a change of the posture as described below. In addition, it isdetermined whether or not host vehicle collides with the mobile objectat an intersection between the travel route of the host vehicle and thepredicted travel route of the mobile object.

The vehicle controller 33 controls a host vehicle along the target routeGenerated by the target route generator 31. The vehicle controller 33calculates a target steering angle and a target speed on the basis ofthe target route. Note that, when the collision predictor 32 predicts acollision between the host vehicle and the obstacle, the vehiclecontroller 33 calculates the target steering angle and the target speedsuch that the host vehicle does not collide with the obstacle. Inaddition, the vehicle controller 33 outputs a target steering torque forimplementing the target steering angle to the steering device 102. Inaddition, the vehicle controller 33 outputs a target engine torque or atarget braking pressure for implementing the target speed to the drivingdevice 103 or the braking device 104.

Now, an operation of the travel control device 1 performed, for example,when the host vehicle travels straightly, by assuming that the hostvehicle passes by the vicinity of a pedestrian will be described withreference to FIGS. 2 and 3. A driver manipulates an automatic drivingbutton (not shown) in advance to start an automatic driving mode.

The image data obtained by photographing the surroundings of the hostvehicle are input from the external environment recognition device 21 tothe surrounding environment recognition unit 22 on a frame basis. Thesurrounding environment recognition unit 22 calculates a travel space230 using the image data input from the external environment recognitiondevice 21. Here, the external environment recognition device 21 is anon-vehicle camera that photographs the surrounding environment in frontof the host vehicle.

FIGS. 3(a) to 3(b) illustrate exemplary images for a forward sight ofthe host vehicle generated by the surrounding environment detectiondevice 2 at timings T=t1, T=t2, and T=t3 on a constant interval basis.In the following description, it is assumed that “T=t3” is the currenttiming, and the subscripts 1, 2, and 3 of the reference signs denoteinformation at the timings T=t1, T=t2, and T=t3, respectively. In theforward sight image of FIG. 3(a), a pedestrian 206 walks straight in theforward left side of the host vehicle 200 in the opposite direction inparallel to the host vehicle.

The pedestrian 206 is detected from the forward sight image from theexternal environment recognition device 21 by applying a patternmatching technique well known in the art to the forward sight image. Inaddition, information regarding the position thereof or the like isobtained. For example, the surrounding environment recognition unit 22recognizes a contour of the pedestrian 206 as shape data obtained bystraightly linking the contour of the pedestrian 206. In addition, thesurrounding environment recognition unit 22 detects a movement directionof the pedestrian 206 from a difference between the images at thetimings T (t1, t2, and t3) and obtains speed vectors V1, V2, and V3representing movement velocities thereof.

In the surrounding environment recognition unit 22, informationregarding the shape of the pedestrian is set in advance, so that thepedestrian is recognized distinguishably from other obstacles such as avehicle. As illustrated in FIGS. 3(a) to 3(c), an approximate bustcenter position B and an approximate head center position H of apedestrian are recognized. In addition, as illustrated in FIG. 3(c), abust direction Db, a head direction Dh, and each line-of-sight directionDe of the pedestrian are recognized. Furthermore, a projected area Ah ofthe head and a projected area Ab of the bust of the pedestrian arerecognized for the external environment recognition device 21. Moreover,a grounded point F between the pedestrian's leg and the ground surfaceis recognized. Furthermore, an inverted angle θ of the bust (an anglebetween a line obtained by linking the approximate center position B andthe approximate grounded point F and a vertical direction) is alsorecognized. The head direction Dh and each line-of-sight direction Demay be estimated on the basis of the projected area A for the externalenvironment recognition device 21, a direction of the line obtained bylinking left and right shoulders, and the like.

The surrounding environment recognition unit 22 sets an allowable space230 where no obstacle exists, for example, on the basis of the forwardsight image illustrated in FIGS. 4(a) and 4(b). The target routegenerator 31 starts a process of generating a target route. The targetroute generator 31 calculates a target route 231 such that the vehicletravels the allowable space 230 detected by the surrounding environmentrecognition unit 22. Note that, since the mobile object such as apedestrian has a size, the target route 231 becomes a band-shaped travelroute having a width. The target route generator 31 expresses a travelroute of a straight interval as a straight line and approximates aturning route by combining a clothoid curve and a circular arc.

The collision predictor 32 determines whether or not the host vehicle200 collides with the obstacle when it moves along the target route 231.The collision predictor 32 calculates a predicted route 400 predicted asthe pedestrian 206 passes on the basis of a movement direction of themobile object, for example, a movement direction of the pedestrian 206detected by the surrounding environment recognition unit 22. Note that,since the mobile object such as a pedestrian has a size, the predictedroute 400 becomes a band-shaped travel route having a width.

FIG. 2(a) is a diagram illustrating an exemplary predicted route 400 ofthe pedestrian 206 generated by the collision predictor 32. In thepredicted route 400 of this example, it is assumed that the pedestrian206 walks straightly and directly as indicated by the speed vector V3.

The collision predictor 32 obtains information regarding the bust centerposition B, the head center position H, the bust direction Db, the headdirection Dh, the line-of-sight direction De, the head projected areaAh, the bust projected area Ab, the grounded point F, and the bustinverted angle θ of a pedestrian from the surrounding environmentrecognition unit 22 as described above. The vehicle control device 3stores at least one previous version of this information in the memoryunit 34. Specifically, in addition to the current timing T=t3,information at the timing T=t2 is stored. In the case of FIG. 2(a),there is no significant change in the body directions Db, Dh, and De orthe inverted angle θ between the current timing T=t3 and the previoustiming T=t2. In this case, the collision predictor 32 predicts that thedirection of the speed vector V3 remains as it is.

The collision predictor 32 calculates a predicted collision intersection401 between the target route 231 and the predicted route 400 as aposition where the host vehicle 200 possibly collides with the obstacle.The collision predictor 32 calculates the target route 231 of the hostvehicle 200 and the time elapsing until each of the host vehicle and thepedestrian arrives at the predicted collision intersection 401 of thepredicted route 400 of the pedestrian 206, and determines whether or notthe host vehicle 200 and the pedestrian 206 collide with each other onthe basis of a positional relationship when each of the host vehicle andthe pedestrian arrives at the predicted collision intersection 401. Inthe case of FIG. 2(a), since there is no predicted collisionintersection 401 between the target route 231 and the predicted route400, the collision predictor 32 outputs a determination result that thehost vehicle 200 does not collide with the pedestrian 206 to the vehiclecontroller 33. In this case, the vehicle controller 33 guides the hostvehicle 200 along the target route 231 generated by the target routegenerator 31. In this case, the vehicle controller 33 determines thetarget speed and the target steering angle such that the host vehicle200 moves along the target route 231. In addition, the vehiclecontroller 33 outputs the target steering angle to the steering device102 and outputs the target speed to the driving device 103 and thebraking device 104. However, considering a case where the pedestrianabruptly approaches the target route 321, such as a case where thepedestrian turns over, the vehicle controller 33 may decelerate the hostvehicle 200. For the similar reason, the vehicle controller 33 mayperforms the calculation again such that the target route 231 becomesdistant from the pedestrian.

Next, a case where it is determined that the host vehicle 200 collideswith the pedestrian 206 will be described with reference to FIGS. 3(b)and 4(a). It is assumed that the surrounding environment recognitionunit 22 outputs information regarding the image data and the pedestrianas illustrated in FIG. 3(b).

When a pedestrian changes a travel route, a change symptom is exhibitedstarting from a part having a small physical load. Initially, adirection of the body changes in order of the line-of-sight directionDe, the head direction Dh, the bust direction Db, and the entire bodydirection. In addition, when a pedestrian going straight changes thedirection, the inverted angle θ of the bust also changes before thewalking speed changes. Therefore, it is possible to predict a travelroute change of the pedestrian more accurately at the earlier timing byobtaining a change of the direction or angle of each part using thesurrounding environment detection device 2 in addition to theinformation on the walking speed or the walking acceleration.

In the example of FIG. 3(b), at the timing t2, the light-of-sightdirection De2 and the head direction Dh2 are directed to the left morethan the bust direction Db2 or the speed vector V2. Similarly, at thistiming, it is possible to predict a possibility that the pedestrian 206turns to the left. At the timing t3, the bust direction is also directedto the left more than the speed vector V2. In addition, the invertedangle θ3 also changes to the left falling direction. It may bedetermined that the pedestrian 206 turns to the left on the basis ofsuch a change of the direction or the angle.

In this case, the collision predictor 32 determines that the hostvehicle 200 may collide with the obstacle on the target route 321, andthe vehicle controller 33 performs control such that the host vehicle200 is decelerated or stops in some cases before a marginal distance YLfrom the intersection 401 on the forward travel route 300 to avoidcollision with the pedestrian 206. In this case, the target speed is setlower, compared to a case where a determination result that the hostvehicle 200 does not collide with the pedestrian 206 is obtained.

Here, it is desirable to change the marginal distance YL on the basis ofthe travel direction of the pedestrian 206 at the predicted collisionintersection 401. For example, if a case where the speed vector of hostvehicle 200 is perpendicular to the speed vector of the pedestrian 206is compared with a case where they are in parallel to each other, it canbe said that the collision risk is higher when they are perpendicular.For this reason, if the speed vector of the host vehicle 200 and thespeed vector of the pedestrian 206 are perpendicular to each other, itis possible to avoid collision with the pedestrian 206 with a moresufficient margin by securing a longer marginal distance YL and reducingthe target speed of the host vehicle 200, compared to a case where thespeed vector of the host vehicle 200 and the speed vector of thepedestrian 206 are not perpendicular to each other.

In the aforementioned example, in order to avoid the collision, the hostvehicle 200 is decelerated depending on the predicted travel route ofthe pedestrian 206. Alternatively, the target route of the host vehicle200 may be changed instead of the deceleration or in combination withthe deceleration. Specifically, the vehicle controller 33 may performthe calculation again such that the target route 231 becomes distantfrom the pedestrian. However, in this case, the target route 231 is setsuch that the host vehicle 200 becomes distant from the pedestrian 206,compared to a case where a determination result that the host vehicle200 does not collide with the pedestrian 206 is obtained.

Note that the determination result of the collision predictor 32 may benotified to a driver using a sound generator device 105 or the displaydevice 106 without controlling the vehicle 200. In this case, control isperformed such that the sound generating timing of the sound generatordevice 105 is expedited, and the volume increases as the direction orangle of the pedestrian 206 changes to increase the collision risk.Alternatively, control is performed such that the timing of displayingon the display device 106 is expedited, the display luminance increases,and the display area increases as the direction or angle of thepedestrian 206 changes to increase the collision risk.

FIGS. 2(b) and 4(b) illustrates an exemplary case where the pedestrian206 walks to approach the host vehicle 200. In the case of FIG. 2(b), achange is insignificant in each of the body directions Db, Dh, and De orthe inverted angle θ between the current timing T=t3 and the previoustiming T=t2. In this case, the collision predictor 32 predicts that thedirection of the speed vector V3 remains as it is. It is predicted thatthe pedestrian 206 enters the travel space 230 of the host vehicle.Meanwhile, in the case of FIG. 4(b), it is predicted that the headdirection Dh or the bust direction Db changes in parallel to the targetroute 231 of the host vehicle 200. Therefore, the vehicle controller 33reduces the target speed in the case of FIG. 2(b), compared to the caseof FIG. 4(b). Alternatively, the target route generator 31 changes thetarget route to the left.

As described above, the travel control device 1 obtains a change of thetravel route of the pedestrian with high accuracy using a change of theposture or a change of the line of sight of the pedestrian, inparticular, using a change of the body direction or a change of theinverted angle to appropriately control the vehicle travel. As a result,even when a pedestrian makes a change of the travel route morecomplicatedly than a constant movement or a constant speed movement, itis possible to predict a motion of the pedestrian at an earlier timingwith high accuracy. In addition, as it is predicted that the traveldirection of the pedestrian and the travel direction of the host vehicle200 approach each other perpendicularly on the basis of a change of theposture or the line of sight of the pedestrian, the speed of the hostvehicle 200 is controlled to be lower. As a result, it possible toprovide a travel control device capable of implementing a safer travelcontrol.

REFERENCE SIGNS LIST

-   1 travel control device-   2 surrounding environment detection device-   3 vehicle control device-   21 external environment recognition device-   22 surrounding environment recognition unit-   31 target route generator-   32 collision predictor-   33 vehicle controller-   34 memory unit-   102 steering device-   103 driving device-   104 braking device-   105 sound generator device-   106 display device-   200 host vehicle-   206 pedestrian-   230 allowable space-   231 target route-   400 predicted route-   401 predicted collision intersection-   A projected area for sensor-   B bust position-   De line-of-sight direction-   Db bust direction-   Dh head direction-   F grounded point-   H head position-   V speed vector-   θ inverted angle

The invention claimed is:
 1. A travel control device that enables safertravel control of a vehicle comprising: a memory that stores a previousbody angle of pedestrians; and a processor that is; communicativelycoupled with the memory and the camera, wherein the processor isconfigured to: obtain, using the camera, a first image of a pedestrianin a first time period, determine, using the memory, a first center ofthe pedestrian based on the first image, determine, using the memory, afirst grounded point between legs of the pedestrian based on the firstimage, calculate a first inverted angle between a line obtained bylinking the first center and the first grounded point in a verticaldirection, receive, using the camera, a second image of the pedestrianin a second time period, determine, using the memory, a second center ofthe pedestrian based on the second image, determine, using the memory, asecond grounded point between the legs of the pedestrian based on thesecond image, calculate a second inverted angle between a second lineobtained by linking the second center and the second grounded point inthe vertical direction, predict a travel route of the pedestrian basedon a change between the first inverted angle and the second invertedangle, control motion of the vehicle based on the travel route of thepedestrian predicted.
 2. The travel control device according to claim 1,wherein the processor is further configured to: predicts a change of atravel direction of the pedestrian from a change of the body angle ofthe pedestrian, and control the vehicle such that a speed of the vehicleis decreased when the change of the travel direction results in a pathof the vehicle and the travel direction of the pedestrian areperpendicular to each other.
 3. The travel control device according toclaim 2, wherein a bust direction of the pedestrian defines the bodyangle.
 4. The travel control device according to claim 3, wherein theprocessor is further configured to: obtains a head position of thepedestrian from a projected area of the head of the pedestrian withrespect to a detection direction of the camera, and obtains the bustdirection of the pedestrian using a projected area of the bust of thepedestrian with respect to the detection direction of the camera.
 5. Thetravel control device according to claim 1, wherein the processor isfurther configured to: detects a line of sight direction of thepedestrian and a head position of the pedestrian based on the firstimage, the memory stores the line of sight of the pedestrian and thehead position of the pedestrian, and predicts the travel route of thepedestrian using a change of the line of sight direction of thepedestrian and a change of the head position of the pedestrian.
 6. Thetravel control device according to claim 1, wherein the processor isfurther configured to: control the vehicle such that a speed of thevehicle is decreased when the second inverted angle of the pedestrian ischanged to a pedestrian falling direction.